Effect of varying the case mix on the standardized mortality ratio and W statistic: A simulation study.

OBJECTIVE To evaluate the validity of using the standardized mortality ratio (SMR) and the W statistic as risk-adjusted measures of hospital mortality to judge ICU performance. DESIGN APACHE (acute physiology and chronic health evaluation) II data were collected prospectively from the surgical ICU (SICU) at a single institution using all adult admissions (n = 6806) over an 8-year period (excluding cardiac surgical patients, burn patients, and patients under 16 years of age). Using a computer simulation technique, virtual ICUs (VICUs) with mortality rates between 5% and 16% were constructed. After first dividing the original data set into deciles of risk, each VICU was constructed by randomly resampling between 10 and 680 patients from each decile. The SMR, W statistic, and Z statistic were calculated for 10,000 different case mixes. SETTING The SICU at a 450-bed teaching hospital. PATIENTS A group of 6,806 adult patient admissions, excluding cardiac surgical patients and burn patients. MEASUREMENTS AND RESULTS VICUs were created from a data set of actual patients treated at one institution in order to test the hypothesis that the SMR and W statistic would remain invariant when applied to subsets of patients from a single institution. Instead, the SMR and W statistic were found to be very sensitive to changes in case mix. The SMR and W statistic were linear functions of the simulated ICU mortality rate. CONCLUSION This simulation demonstrates that the SMR and the W statistic based on APACHE II cannot be used to compare outcomes of ICUs. We have proposed a revision of the SMR that eliminates the effect of case mix and allows for more accurate comparisons of ICU performance.